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Summary 

This study explores the idea of providing 
cockpit automation training to airline- 
bound pilots using advanced automation 
equipment now commonly found in small 
piston training airplanes. This idea takes 
advantage of the striking similarity 
between the small-airplane cockpit 
automation systems and those found in 
popular jet transport airplanes. Two 
curricula are described designed to teach 
cockpit automation in airplanes big and 
small, exploiting the overlap between the 
two. Two experiments were conducted to 
evaluate the feasibility and effectiveness of 
this kind of “bridge” training that allows 
students to get hands-on learning and 
experience with cockpit automation early 
in their pilot training. In a first experiment, 
pilots mastered a set of tasks and 
maneuvers of varying difficulty using a 
small airplane GPS navigation computer, 
autopilot, and flight director system. 
Students were then tested on their ability to 
complete a similar set of tasks and 
maneuvers using a computer-based 
simulator of the flight management and 
guidance systems found in a popular jet 
transport aircraft. Pilots attempted the jet 
transport tasks with no prior exposure to 
the equipment, no training, and no 
reference materials. Pilots were told to try 
to apply the principles they had learned in 
the small airplane. The results indicate a 
high degree of success: pilots were able to 
successfully complete 77% of all tasks in 
the jet transport on their first attempt. An 
analysis of a control group that received 
no small airplane automation training 
suggests that the pilot trainees’ success was 
attributable to the application of 
automation principles they had learned, 
rather than superficial strategies guided by 
words and labels that appear on the knobs 
and buttons of the automation equipment. 
A second experiment looked at two 
different ways of delivering small-airplane 


cockpit automation training: (1) a self- 
study method in which pilots were 
assigned readings in advance and were 
then evaluated in flight; and (2) a dual 
ground instruction method in which pilots 
received one-on-one instruction 
immediately prior to each flight. The 
results showed a slight advantage for the 
self-study method. Overall, the results of 
the two studies cast a strong vote for the 
incorporation of cockpit automation 
training in curricula designed for pilots 
who will later transition to the jet fleet. 

Introduction 

Among the challenges of transitioning 
from small piston training airplanes to the 
modem jet fleet is the requirement of 
learning to use cockpit automation. Airline 
carriers continue to struggle with training 
pilots transitioning from the world of 
general aviation training, or from non- 
glass cockpit equipped aircraft (refs. 1 and 
2). Studies of cockpit automation use 
continue to point to areas in which 
automation training should be improved 
(refs. 3, 4, 5, 6, and 7). Although the 
Federal Aviation Regulations (FAR) 
contain specific aeronautical knowledge 
and flight experience requirements for 
other topics such as aerodynamics, 
weather, regulations, and even other 
aircraft systems, there are no such 
requirements for this emerging and critical 
component of pilot skill. Consequently, it 
is typically the case that pilots come to 
initial job training with little or no 
experience with cockpit automation. 

This work aims to bridge the gap between 
efforts to train future professional pilots 
and airline carrier training by taking 
advantage of the advanced cockpit 
automation that is now available in small 
training airplanes. Using modem GPS 
navigation computers, autopilots, and 
flight director systems available in piston 


training airplanes, a cockpit automation 
curriculum has been designed that teaches 
fundamental cockpit automation concepts 
and skills to the student pilot. This 
curriculum has been designed to match a 
second curriculum aimed at teaching 
cockpit automation skills in jet transport 
airplanes. Taken together, the two 
curricula provide a simple, low-cost 
solution to the problem of teaching these 
important skills. 

This Technical Memorandum briefly 
describes the two curricula and reports the 
results of two empirical studies. In a first 
study, student pilots completed the cockpit 
automation training curriculum in a 
piston-engine training airplane. Upon 
completion of this training, pilots were 
then asked to demonstrate the same set of 
skills using a computer-based simulation 
of the cockpit automation suite found in a 
popular jet transport airplane. Pilots’ 
performance during the small-airplane 
cockpit automation training and their 
performance with the jet transport 
simulation were recorded and analyzed. In 
a second study, two alternatives for 
delivering small-airplane cockpit 
automation training were considered: one- 
on-one instruction with a ground 
instructor, and a self-study program in 
which pilots read written materials on their 
own time. 

The results of the two studies clearly 
indicate that time spent learning about and 
gaining experience with cockpit 
automation in piston training airplanes 
pays large dividends when later confronted 
with the task of mastering automation 
found in the jet fleet. 


A Big Airplane Cockpit 
Automation Training 
Curriculum 

Casner (ref. 8) describes a detailed 
curriculum covering basic and 
intermediate cockpit automation skills and 
concepts needed to work proficiently in 
the modem airline cockpit. The 
curriculum uses the flight management 
computer, autopilot, and flight director 
systems found in most modem jet 
transports (see figure 1) to teach skills and 
concepts required to work cooperatively 
and proficiently with automation when 
performing the traditional tasks of flight 
navigation, guidance, and control. 

The program of concepts and skills 
contained in the curriculum benefits from 
almost twenty years of laboratory and field 
research, along with inputs gathered from 
airline training departments, avionics 
manufacturers, and individual line pilots, 
instructors, and check airmen who have 
accumulated much experience teaching 
and using cockpit automation. 

The curriculum goes beyond previous 
efforts to prescribe proficiency standards 
for cockpit automation in that it contains 
more than skills or procedures that must 
be memorized by rote and demonstrated 
by the student. This curriculum requires 
that the student understand the underlying 
principles of how the automation works 
and how the flight crew and automation 
work together as a team. Particular 
emphasis is placed on how the role of the 
flight crew is changed when automation is 
used, the ways in which the system of 
flight crew and automation can break 
down, and strategies for delegating work 
among flight crew and automation 
equipment. The curriculum described in 
the book sets the mark for the skilled and 
aware pilot in the modern airline cockpit. 
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The lllght management computer (FMC) 
is programmed with the flight route. The 
FMC provides navigational services 
through all phases of the flight. 


Figure 1. Cockpit automation typical of a jet transport aircraft. 




A multifunction display (MFD) can present information 
from different cockpit automation systems. This MFD 
shows a display of surrounding traffic and the aircraft's 
present position. 


Figure 2. Cockpit automation typical of a small piston training airplane. 
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CONCEPT or SKILL 


Planning the Flight Route 


Basics of electronic flight plannin 


The flight navigation computer 


Learning the knobs, dials, and buttons 


Entering the basics of the flight route 


How the navigation computer calculates the details 


Calculating the lateral portion of the route 


Calculating the vertical portion of the route 


Importance of reviewing the flight plan 


Mistakes humans make 


Mistakes computers make 


PISTON 




Following the Flight Route 


Monitoring progress along the route 


Lateral guidance 


Vertical guidance 


Descent plannin 


Precision approaches 


Non-precision approaches 


RNAV approaches 



En Route Modifications 


Direct to 


Diversions 


Departing & Rejoining the Route 


Intercept course 


Intercept leg to 


Intercept radial 


Earlv and late descents 


Maintaining awareness 


Holds 


Procedure turns 


Missed approaches 


Flying with an Autopilot and Flight Director 


Autopilot functions & targets 


The flight mode annunciator 


The flight director 


Autopilot maneuvers 


Climbs and descents 


Headings 


Intercepts 


Armed vs. engaged 


Approaches 


Disconnecting the autopilot 


Mode awareness and confusion 


Figure 3. Common elements of the cockpit automation curricula. 
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A Small Airplane Cockpit 
Automation Training 
Curriculum 

Casner (ref. 9) describes a second cockpit 
automation curriculum designed to teach 
the same set of concepts and skills using 
cockpit automation systems now common 
in many small training airplanes. In the 
place of the flight management computer, 
autopilot, and flight director systems 
found in jet transport airplanes, this 
curriculum makes use of the GPS 
navigation computers, autopilots, and 
flight director systems (see figure 2) found 
in small training airplanes. 

Although there are many cosmetic 
differences between the two systems, the 
underlying operating principles, as well as 
the human factors issues of working with 
computers in the cockpit, are 
fundamentally the same. 

Comparing the Two Cockpit 
Automation Curricula 

Figure 3 summarizes the elements of the 
two cockpit automation curricula, 
highlighting the similarities between the 
two. 

Figure 3 shows that most of the concepts 
and skills covered in the jet transport 
airplane curriculum are also covered in the 
small airplane curriculum. The small 
airplane curriculum presents the student 
with opportunities to develop a range of 
cockpit automation skills, and to gain 
hands-on experience with the challenging 
job of performing cockpit duties in 
concert with sophisticated automation 
systems. 

EXPERIMENT 1 

The aim of the first experiment was to 
answer the most basic question about 


cockpit automation training in airplanes 
big and small: To what extent can cockpit 
automation concepts and skills acquired in 
a small piston training airplane be 
successfully transferred to the operation of 
the cockpit automation systems found in a 
jet transport aircraft? 

In this experiment, we used a simulation of 
a popular jet transport airplane to compare 
the performance of two groups of pilot 
participants. One group of pilots 
completed a cockpit automation training 
program taught in a small piston training 
airplane. A control group of pilots 
received no such training. 

Method 

Participants 

Sixteen commercial instrument rated pilots 
were recruited from local professional 
flight training schools. Pilots ranged from 
300 to 1,600 hours of flight experience 
with a mean of 1 106 hours. Pilots were 
told they would not be paid for their 
participation but would receive instrument 
flight experience using cockpit 
automation. 

Procedure 

The sixteen pilots were divided randomly 
into two groups. The experimental group 
would work through the small airplane 
cockpit automation curriculum and then 
be tested using the jet transport computer- 
based simulator. A control group would be 
tested on the computer-based simulator 
first, without the small-airplane cockpit 
automation training. The control group 
would later receive a portion of the small- 
airplane cockpit automation training, but 
their training or performance was not 
recorded as part of the experiment. 

The purpose of the control group was to 
factor out any successes that might be 
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enjoyed due to what Irving, Poison, and 
Irving (ref. 10) refer to as label following. 
Label following occurs when a computer 
system provides simple cues about how it 
might be operated, typically in the form of 
labels that suggest the purpose or 
operation of knobs, buttons, and dials on 
the equipment. When using label 
following, operators can often succeed in 
completing a task without any knowledge 
or skill related to that task. For example, 
consider the task of calling up the Index 
page on a control display unit (CDU). A 
person with little or no knowledge about 
cockpit automation might notice the 
button labeled INDEX, shown on CDU in 
figure 4, and correctly hypothesize that 
pushing this button will accomplish the 
task. 

Although label following cues are 
legitimate components of expert 
knowledge, we would like to distinguish 
between success attributable to true 
understanding of the system, and success 
due to label following. 



Figure 4. An example of a task that offers 
label following. 


The small airplane cockpit automation 
training program 

For the eight pilots participating in the 
treatment group, the small airplane cockpit 


automation training occurred in five 
scheduled sessions. Prior to each session, 
each pilot was assigned a chapter to read in 
the small airplane cockpit automation 
book (ref. 9) Pilots were told to master the 
material as best as they could, and that 
during the upcoming session, they would 
have the opportunity to demonstrate and 
practice their newly learned skills in flight. 
It was emphasized that pilots should 
attempt to master the skills such that they 
could demonstrate them without the need 
for intervention by the experimenter, 
although intervention would be available if 
needed. 

During each session, the experimenter 
briefly reviewed the skills that would be 
covered during the flight, provided the 
pilot with charts covering the routes and 
approaches to be flown, and answered any 
questions the pilot had about the reading. 
The airplanes used for the flights 
contained the same GPS navigation 
computer, autopilot, and flight director 
system described in the cockpit 
automation book read by the pilots. 

During the flight, the experimenter rode in 
the right seat and did not operate the 
controls. A script for each flight was 
prepared in advance and used by the 
experimenter to ensure that each flight 
proceeded in accordance to a set plan, and 
that each pilot was presented with the same 
set of scripted tasks. A palmtop computer 
was used to record any interventions 
required by the experimenter for each 
task, errors made by the pilot on any task, 
or assistance requested by the pilot for any 
task. A scorecard was kept for each pilot 
and flight. For each task, if the pilot was 
able to complete the task with no 
intervention on the part of the 
experimenter, the pilot received a score of 
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Flight 1 

Check navigation database 
Enter waypoints and procedures 
Review route 

Monitor active waypoint and progress 
Plan a descent w/crossing restriction 
Direct to 

Add and delete waypoints 
GPS approach to minimums 


Flight 3 

Missed approaches 
Holds 

Flight 4 

Autopilot: Heading 

Autopilot: Constant-rate climbs and descents 
Autopilot: Intercepts 


Flight 2 

Intercept course 
Vectored GPS approaches 


Flight 5 

Check proficiency on all maneuvers 


Figure 5. Breakdown of the five small airplane cockpit automation training flights. 


1. If an intervention of any form, 
regardless of how subtle (e.g., words, 
gestures, sounds), was required, a score of 
0 was recorded for that task. Appendix 1 
presents the complete script of tasks for 
each of the five sessions. 

The topics introduced during the five 
flight sessions are summarized in figure 5. 
It is important to note that the first four 
flights gradually introduce new skills, 
while providing opportunity to practice 
skills learned on the previous flights. The 
fifth flight was intended as a “check” 
flight. No new skills were introduced and 
the aim was to measure the pilots’ current 
level of proficiency. 

The jet transport simulator evaluation 

Following the conclusion of the small 
airplane training sessions, all sixteen pilots 
participated in a test session in which they 
were asked to perform a series of tasks 
using a computer-based simulation of the 
cockpit automation systems found in a 
popular jet transport airplane. Eight of the 
pilots had received the small airplane 
cockpit automation training and eight had 
not. It was explained that pilots would 
receive no training on the jet transport 


systems or have the opportunity to access 
any reference materials for the systems. 
The aim of the study was to determine to 
what extent their existing knowledge could 
help guide them through the tasks. The 
treatment group had their instrument 
flying skills together with their small- 
airplane cockpit automation training. The 
control group had their instrument flying 
skills to guide them, together with any 
label following cues present on the jet 
transport automation equipment. 

During the jet transport systems session, 
the same data collection procedure was 
used. Pilots were presented with tasks and 
asked to do their best to perform them 
without asking for intervention from the 
experimenter. If an impasse was 
encountered, pilots could ask for 
intervention, these interventions were 
recorded, and a score of 0 was recorded 
for that task. As in Experiment 1, if a pilot 
was unsuccessful on a particular task, the 
experimenter demonstrated the task before 
moving on to the next task. Since the jet 
transport travels as much as five times 
faster than the piston airplane, the 
simulation was frozen while the 
experimenter took the time to provide the 
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Pilot 


Figure 6. Percentage of tasks completed correctly for the individual pilots in 
the training and no training groups. 


needed interventions. A scorecard similar 
to the one used during the cockpit 
automation training was used to record 
interventions made by the experimenter. 
The complete script of tasks for the jet 
transport sessions is presented in Appendix 
2 . 


Results and Discussion 

Overall Performance 

A first question posed by the experiment is 
the extent to which the small-airplane 
cockpit automation training and 
experience leveraged pilot performance 
when presented with the jet transport 
airplane automation. Figure 6 shows a 
graph of the percentage of tasks 
completed correctly by each pilot using 
the jet transport automation. The dots in 
figure 6 represent individual scores (on all 
tasks combined) for the sixteen pilots. The 
pilots who received the small airplane 
cockpit automation training performed 


significantly better than the control group 
(df = 14. t = 6.23, p < .001). 

The overall performance of the 
experimental group casts a vote for the 
usefulness of cockpit automation training 
in small airplanes. These pilots were able 
to successfully perform 77% of all tasks 
on the jet transport airplane on the first 
try. 

Success Due To Label Following 

The mean success rate of 54% for the 
control group prompts the question of to 
what extent was their success attributable to 
superficial label following. To answer this, 
tasks were divided into two groups, those 
for which label cues appeared on the 
equipment, and those for which no cues 
appeared. The graph in figure 7 shows the 
results for the experimental and control 
groups on label-cued and non-label-cued 
tasks. 
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Figure 7. Breakdown of scores for tasks completed with and without presence of 
label cues. 


A 2-way analysis of variance reiterated the 
main effect of the advantage due to 
receiving the small-airplane cockpit 
automation training (F=67.5, p < .001), a 
main effect due to the presence of label 
cues (F=44.5, p < .001), and a significant 
interaction between the two factors 
(F=25.1, p < .001). 

For the pilots who received the small- 
airplane cockpit automation training, there 
was no significant difference between the 
two task types, suggesting that the cues 
provided by their knowledge were as 
strong as the cues provided by the labels. 
The pilots who received no cockpit 
automation training performed well when 
label cues were present but poorly in the 
absence of label cues. This suggests that 
their success occurred in the absence of 
understanding of how to operate the 
systems. Lastly, the pilots who received the 
small-airplane cockpit automation training 
performed significantly better on tasks for 
which label following was possible, than 
did their control group counterparts. This 
suggests that the training group imparted 
knowledge on tasks even when label cues 


were present, and this knowledge led to 
significantly greater performance. 

Breakdown by Procedure 

A second interesting way to look at pilot 
performance is to consider the particular 
tasks that pilots were asked to perform 
using the jet transport automation. Since 
the list of tasks is quite lengthy (see 
Appendix 2) and many of the tasks are 
related, figure 8 organizes the tasks into 
groups that pilots typically refer to as 
procedures. A procedure is defined as a 
collection of tasks that lead to the 
accomplishment of a goal. For example, 
the Enter waypoints and 
procedures procedure is composed of 
six tasks that accomplish the goal of 
entering a flight route into the flight 
computer. 

When looking at the data in figure 8, it is 
important to note that the scores for the 
procedures listed in figure 8 represent the 
scores recorded for all of the component 
tasks in each procedure. For example, if a 
pilot successfully completed the first three 
tasks in the Enter waypoints and 
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procedures procedure but failed to 
complete the last three, a score of .5 would 
result. 

As expected, pilots performed best on 
tasks that resembled tasks that they had 
learned during their small airplane cockpit 
automation training. For example, nearly 
all pilots completed the Enter en 
route waypoints and 
procedures and Monitor active 
waypoint and progress 
procedures. These procedures, along with 
their associated concepts, were nearly 
identical to the ones learned in the small 
airplane. Other procedures, such as 
Direct to and Check navigation 
database, were similar but not identical. 
Pilots experienced high degrees of success 
on these procedures. Some procedures, 


such as Position initialization 
and Execute modifications, were 
completely absent from the small airplane 
equipment and training. Pilots had little 
success in completing these procedures. 

The most encouraging result is the 
intercept course procedure. Previous 
studies with experienced airline pilots have 
shown this task to be difficult (ref. 10). 

The intercept course procedure combines 
several advanced concepts such as the 
notions of departing and rejoining the 
planned route, and armed vs. engaged 
autopilot modes. Slightly less than 70% of 
Irving et al’s airline pilots who had just 
completed an airline initial training course 
on a Boeing 737-300 were able to 


Task 

Trained 

Group 

Control 

Group 

t-Test 

Check navigation database 

.88 

.25 

p < .01 

Position initialization 

.58 

.54 

No 

Enter waypoints and procedures 

.94 

.81 

No 

Review route 

.69 

.56 

No 

Execute modifications 

.25 

.25 

No 

Monitor active waypoint and progress 

1 

.94 

No 

Direct to 

.88 

.75 

No 

Add and delete waypoints 

.79 

.37 

p < .05 

Hold 

.81 

.63 

No 

Enter crossing restriction 

.63 

.5 

No 

Explain purpose of entering crossing restriction 

.75 

.13 

p < .01 

Constant-rate climbs and descents 

.66 

.13 

p < .001 

Heading 

.75 

.75 

No 

Intercept course 

.75 

.25 

p < .001 

Constant-speed climbs and descents 

1 

1 

No 


Figure 8. Percentage of procedures successfully completed by pilots who did and did not 
receive small-airplane cockpit automation training. 
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successfully complete this procedure 
following explicit training using the same 
equipment used for the test. The 
experimental group described here 
completed the task successfully 75% of the 
time. There is reasonable evidence to 
suggest that this was due to the emphasis 
the automation materials place on 
conceptual understanding of the task. In 
the cockpit automation textbook, pilots are 
taught to ask themselves two questions that 
are promised to guide them in any 
advanced route modification situation: (1) 
Where am I going? and (2) How am I 
going to get there? One pilot foundered 
on the procedure for about thirty seconds 
and then spontaneously verbalized the two 
questions. The pilot quickly assembled a 
procedure that successfully solved the 
problem. 

Two procedures unexpectedly tripped up 
roughly half of the pilots. One was the 
constant-rate descent procedure. This 
procedure is almost identical to the one 
used in the small-airplane automation, and 
one for which most pilots had 
demonstrated mastery. When pilots were 
given the first step in the procedure, they 
were generally able to complete the 
remaining steps immediately. 

A second procedure that challenged 
subjects was the Enter crossing 
restriction procedure. The solution 
for this procedure is also somewhat similar 
to the solution used on the small-airplane 
automation. 

Correlating Total Flight Time and 
Performance with Cockpit 
Automation 

A last interesting analysis is to look at the 
relationship between the total flight 
experience of each pilot participant and 
their performance with the jet transport 
cockpit automation. For the group that 
received the small airplane cockpit 
automation training, the correlation was 


-0.36. For the group that did not receive 
the small airplane training, the correlation 
was 0.15. These results suggest that there is 
little link between total flight time and 
mastery of cockpit automation. Stated in 
another way, total flight time does not 
appear to serve as a substitute for training 
and experience with cockpit automation. 
Cockpit automation proficiency appears to 
be a unique set of skills that must be 
learned in addition to basic airmanship. 


EXPERIMENT 2 

After demonstrating the usefulness of 
providing small airplane cockpit 
automation training, a second experiment 
was designed to examine the differences 
between delivering the small airplane 
cockpit automation instruction to students 
in two different ways. In one condition, the 
material was presented to students in the 
form of a book to be read in their own 
time, as was done in Experiment 1. In a 
second condition, the material was 
presented to students in a traditional one- 
on-one ground instruction scenario. These 
conditions were designed to represent two 
ways in which pilots might learn about 
cockpit automation: (1) as part of a 
program of instruction at a flight school; 
or (2) on a self-study basis as is frequently 
done for many aviation topics. 

For this experiment, a different airplane 
was used that contained a different 
manufacturer’s GPS navigation computer 
and autopilot. Therefore, the condition in 
which material was presented to students in 
book form was replicated to control for 
any differences that might exist between 
the two airplanes. Having two data sets for 
which students used two different kinds of 
navigation computers would also permit an 
informal comparison between the two 
kinds of navigation computers. 
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Method 

Participants 

Sixteen commercial instrument-rated pilots 
were recruited from local professional 
flight training schools. Pilots ranged from 
120 to 3,700 hours of flight experience, 
with a mean of 790 hours. Pilots were told 
they would not be paid for their 
participation but would receive instrument 
flight experience using cockpit 
automation. 

Procedure 

All sixteen pilots received the small 
airplane cockpit automation training, 
individually, in the same five scheduled 
sessions used in Experiment 1. The sixteen 
pilots were divided randomly into two 
groups prior to the first scheduled session. 

The Self-Study group received a version 
of the cockpit automation book described 
in Experiment 1. This version of the book 
used a different manufacturer’s GPS 
navigation computer to explain the same 
cockpit automation curriculum. In the 
same manner as Experiment 1, these pilots 
were assigned readings in the cockpit 
automation book prior to each session. 
Pilots were told to master the material as 
best as they could, and that during the 
upcoming session, they would have the 
opportunity to demonstrate and practice 
their newly learned skills in flight. It was 
emphasized that pilots should attempt to 
master the skills such that they could 
demonstrate them without the need for 
intervention by the experimenter, although 
intervention would be available if needed. 

The Dual-instruction group was told to do 
nothing to prepare for the flight sessions. 
These pilots were told that the experiment 
would cover all of the concepts and skills 
needed for each flight during a dual 
ground instruction session immediately 


prior to the flight. Pilots were told that the 
experimenter would answer any questions 
the pilot might have to bring them to the 
level at which the pilot felt he or she could 
demonstrate the skills without the need for 
intervention by the experimenter, although 
intervention would be available if needed. 

For both groups, prior to each flight, the 
experimenter briefly reviewed the skills 
that would be needed during the flight, 
provided the pilot with charts covering the 
routes and approaches to be flown, and 
answered any questions the pilot had about 
the material. 

During the flight, the experimenter rode in 
the right seat and did not operate the 
controls. The same script used for 
Experiment l was used by the 
experimenter to ensure that each flight 
proceeded in accordance to a set plan, and 
that each pilot was presented with exactly 
the same tasks. A palmtop computer was 
used to record any interventions required 
by the experimenter for any task, errors 
made by the pilot on any task, or 
assistance requested by the pilot for any 
task. A scorecard was kept for each pilot 
and flight. For each task, if the pilot was 
able to complete the task with no 
intervention on the part of the 
experimenter, the pilot received a score of 
1. If an intervention of any form, 
regardless of how subtle (e.g., words, 
gestures, sounds), was required, a score of 
0 was recorded for that task. 

Results and Discussion 

The mean and standard deviation for the 
Self-study and Dual-instruction groups 
were: 0.981 (.023) and 0.91 (.073), 
respectively. A two-tailed t-test yielded a 
significant difference between the two 
groups (df = 14, t = 2.62, p < .05). 

A small advantage was observed for the 
Self-study group. Even though the Dual- 
instruction group had the benefit of 
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Figure 9. Percentage of tasks successfully 
manufacturers’ navigation computer. 

having the procedures explained and 
demonstrated on a one-on-one basis, the 
Self-study group performed slightly 
better. There are a number of plausible 
explanations for the advantages of written 
material over verbally-delivered material 
that have been observed in previous studies 
of learning. First, users of written material 
are able to control the pace of instruction: 
they can proceed to a new sentence, 
paragraph, or topic when they decide they 
are ready. Second, users of written material 
have a persistent record of the instructional 
material that they can review as much as 
they wish. Listeners must rely on 
handwritten notes to record information 
they feel they may need in the future. It is 
interesting to note that none of the pilots 
in the Dual-instruction group made use of 
notes. 


completed by different pilots using different 


Comparing the Two Manufacturers’ 
Navigation Computers 

A direct comparison of the navigation 
computers used in Experiments l and 2 is 
difficult for a number of reasons. First, the 
airplane used in Experiment 2 contained a 
much more sophisticated autopilot, and 
this autopilot played an important role 
during flights 4 and 5. Therefore, a direct 
comparison of data including these two 
flights would be meaningless. We could 
look at the data for the first three flights 
only, and examine the success rates as 
pilots worked toward mastery of the two 
computers. Figure 9 shows the mean 
scores for the proportion of navigation 
computer programming tasks completed 
correctly by the treatment group from 
Experiment 1, and the Self-study group 
from Experiment 2. 

Three t-tests showed a significant 
difference for the two navigation 
computers for Flight 1 (df=14, t=2.06, p < 
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0.05), and for Flight 2 (df=14, t=3.43, p < 
0.01), but no significant difference for 
Flight 3. 

These data suggest that pilots had a more 
difficult time mastering one computer than 
the other, and this result agrees with the 
experimenter’s experience in the cockpit 
with the pilot participants. Although 
neither navigation computer is more 
sophisticated than the other, one of the 
computers contains design features that are 
inconsistent with human factors principles 
that are known to lead to errors and longer 
learning times. A principal problem with 
the navigation computer used in 
Experiment 2 was a lack of consistency 
among conventions used in the user 
interface. The navigation computer used in 
Experiment 2 sometimes requires the user 
to leam different procedures for 
accomplishing the same task under 
different circumstances. In other 
circumstances, this computer leaves the 
user with ambiguous displays that provide 
few clues about how to proceed to the next 
step in a procedure. These design features 
required pilots to memorize work-arounds 
for these situations, and a few flights were 
required before pilots reached proficiency. 


Conclusion 

A number of conclusions can be drawn 
from the research described above: 

Cockpit Automation Skills Learned 
in Small Airplanes Transfer to Big 
Airplanes 

A relatively small investment made in 
acquiring basic skills and experience with 
cockpit automation, now readily available 
to most student pilots, can have a 
tremendous impact on the readiness of that 
pilot when later confronted with more 
sophisticated cockpit automation. The 
demonstrated usefulness of cockpit 
automation found in small training 


airplanes appears to provide a simple, cost- 
effective way of introducing cockpit 
automation to pilots who are still in the 
formative phases of their professional 
aviation careers. This should greatly 
alleviate the problem of new-hire pilots 
arriving to airline initial training programs 
with little or no cockpit automation 
experience. 


It Is Important to Teach Concepts 
Along With Skills 

A principle lesson learned during this 
research is the value of teaching 
underlying principles of cockpit 
automation and automation use in addition 
to teaching button-pushing procedures. It 
must be reiterated that, in Experiment 1 
described above, neither group received 
training on procedures required to 
complete the jet transport airplane tasks. 
Furthermore, looking at figures 1 and 2, 
we can see that the knobs, dials, and 
procedures used to operate the devices 
found in each airplane are quite different. 
The success of the group who received the 
small airplane cockpit automation training 
can only be attributed to the learning and 
application of generalized concepts and 
principles acquired during their training 
using different automation equipment. 

This result is consistent with previous 
studies that have demonstrated that 
teachings focused on knobs, dials, and 
procedures result in fast training times, but 
also tend to result in brittle skills that are 
typically not transferable to other 
equipment, or problems and situations that 
are different from those learned during 
training. Teaching rote procedures helps 
students leam specific procedures quickly. 
However, if the equipment or situations 
encountered in the real world differ from 
those taught in the classroom, and 
challenge the student in new ways; expect 
poor results. Alternatively, training that 
attempts to provide the learner with 
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procedures couched in deeper 
understanding often avoids the limitations 
suffered by “knobs and dials” training. 
Kieras and Bovair (ref. 11) demonstrated 
how students who received “how it 
works” explanations for a set of 
procedures they had learned were 
significantly more successful when 
presented with related problems that 
challenged them in different ways. 
Pennington et al (ref. 12) conducted a 
similar study. Chi et al (ref. 13) looked 
specifically at how students generated and 
successfully used their own “self- 
explanations” while solving problems. 

This study demonstrates again how an 
appropriately-presented skill set can be 
transferred and applied to new, more 
sophisticated equipment. 

Proficiency with Cockpit Automation 
is a Unique Skill Set That Must Be 
Learned 

It appears that proficiency with cockpit 
automation is a separate set of skills to be 
acquired. Having extensive experience in 
airplanes not equipped with cockpit 
automation systems does not appear to be 
a substitute for explicit cockpit automation 
training. Working proficiently with 
advanced computer systems seems to be 
the result of training and experience 
working with advanced computer systems. 

Both Dual-instruction and Self- 
Study/Practice Methods Can Lead to 
Successful Learning 

In the U. S. aviation industry, future 
professional pilots come from a variety of 
training channels. Some pilots receive their 
training as partial fulfillment of a 
university degree program. For these 
pilots, aviation knowledge and skill areas 
that are not part of the required training 
for any FAA certificate or rating are 
introduced in the classroom, as part of 
course work required by the university as 
degree requirements. Other pilots take 


university degrees in other, often related, 
fields, and accomplish their FAA 
certificate flight training on their own at 
local flight schools and training 
academies. These pilots typically learn 
about aviation topics not required as part 
of the FAA certificates and ratings on their 
own, relying on other, more experienced 
pilots, and an expanding market of 
aviation training materials such as books, 
videos, and computer-based simulations. 

The second experiment described above 
demonstrated that an appropriately- 
designed and appropriately-followed 
cockpit automation curriculum can be 
effectively undertaken in either of these 
two popular learning situations. 

Learning Opportunities for Cockpit 
Automation Are Becoming Widely 
Available 

A growing market for learning resources 
for cockpit automation presents a variety 
of opportunities for providing these much- 
needed skills to career-minded student 
pilots. A first category of learning aids are 
the books about cockpit automation that 
are now widely available (refs. 8, 9, 14, 15, 
16, and 17). 

A second type of learning resource are the 
computer-based simulators that provide 
opportunities for hands-on practice. 
Garmin (www.garmin.com) and Bendix- 
King (www.bendixking.com) offer 
computer-based simulations of their GPS 
navigation computers that can be 
downloaded free of charge from their web 
sites. Aerowinx (www.aerowinx.com) 
offers a fully-functional desktop 
simulation of a Boeing 747-400 for $250. 
Lastly, many small training airplanes now 
come equipped with GPS navigation 
computers, autopilots, and flight directors. 
Hands-on access to sophisticated cockpit 
automation systems may be available at 
your local flight school. 
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Appendix 1 


Script of Tasks Used for the Small Airplane Cockpit 
Automation Training Flights 


Flight 1: SQL-027-SQL 

SQL-027 

Program SQL-Sunol-Tracy-ECA-027 on ground 
Announce Sunol 
Program VNAV ECA @ 3,000 
Announce Tracy 
Announce ECA 
Announce Moter 
Announce approach active mode 
Announce Eltro 
Aircraft control 
027-SQL 

Program 027 to SQL on ground 
Insert Tracy and Sunol 
Program diversion 
Look up rwy length and frequency 
Program Sunol to SQL 
Aircraft control 


Flight 2: SQL-MOD-SCK-LVK-SQL 

SQL-MOD 

Program SQL-Sunol-Tracy-Cazli-MOD on ground 
Set OBS 009 to Sunol 
Set GPS to sequencing mode 
Announce Sunol 
Announce Tracy 
Set OBS 018 to Awoni 
Announce Awoni 
Set GPS to sequencing mode 
Announce approach active mode 
Announce Wowar 
Aircraft control 
MOD-SCK 

Program MOD-SCK on ground 
Set OBS 291 to Oxjef 

Set GPS to sequencing mode once established 


17 



Announce Oxjef 
Announce approach active mode 
Announce Ipdew 
Aircraft control 
SCK-LVK 

Program SCK-LVK on ground 

Set OBS 246 to Uhhut 

Set GPS to sequencing mode 

Announce Uhhut 

Announce approach active mode 

Announce Oyahi 

Aircraft control 


Flight 3: SQL-STS-KDV0-069-SQL 

SQL-STS 

Program SQL-STS 
Set OBS 321 to Zijbe 
Set GPS to sequencing mode 
Announce Zijbe 

Announce approach active mode 
Announce Gokuw 
Aircraft control 
STS-DVO 

Program STS-DVO on ground 
Set OBS course to Oriby 
Announce Oriby 
Announce approach active mode 
Announce Eyeji 
Program direct to SGD 
Set OBS 180 to SGD for hold 
Program SGD-069 
Aircraft control 
DV0-069 

Set OBS 268 to Ipary 

Set GPS to sequencing mode when established 
Announce approach active mode 
Announce Ipary 
Aircraft control 


Flight 4: SQL-MRY-WVI-HAF-SQL 

SQL-MRY 

Program SQL-OSI-Sapid-Santy-Mover-SNS-Llynn-MRY on ground 

Engage Heading Select 

Engage VS and arm Altitude Hold 

Set OBS 141 to Sapid 

Arm Nav to capture course 

Set GPS to sequencing mode 
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Announce Sapid 

Engage VS and arm Altitude Hold 
Announce Santy 
Engage Heading Select 
Set OBS 286 to Raine 
Arm Approach to capture course 
Set GPS to sequencing mode when established 
Announce approach active mode 
Announce Raine 
Announce 7.2NM waypoint 
MRY-WVI 

Program MRY-WVI on ground 
Engage VS and arm Altitude Hold 
Set OBS 314 to Dyner 
Arm Approach to capture course 
Set GPS to sequencing mode when established 
Announce approach active mode 
Announce Dyner 
WVI-HAF 

Program WVI-HAF on ground 
Announce Giruc 
Set GPS to OBS mode for hold 
Set GPS to sequencing mode 
Engage Approach to capture course 
Announce approach active mode 
Announce Wohli 

Flight 5: SQL-027-SCK- 1 03-L VK-SQL 

SQL-027 

Program SQL-Sunol-Tracy-ECA-027 on ground 
Announce Sunol 

Engage VS and arm Altitude Hold 

Program VNAV ECA @ 3,000 

Engage VS and arm Altitude Hold 

Announce Tracy 

Set OBS 090 to Moter 

Engage Heading Select and arm Approach 

Set GPS to sequencing mode 

Announce Moter 

Announce approach active mode 

Announce Eltro 

Program direct Wraps 

Use autopilot to accomplish missed approach 
Set OBS 180 Wraps for hold 
Announce Wraps 
Wraps-SCK 

Program Wraps-SCK 
Set OBS 234 to Oxjef 



Engage VS and arm Altitude Hold 
Engage Heading Select and arm Approach 
Set GPS to sequencing mode when established 
Announce approach active mode 
Announce Ipdew 
SCK-103 

Program SCK-103 
Set OBS 285 to Quads for PT 
Use autopilot to accomplish PT 
Announce Quads 

Set GPS to sequencing mode inbound to Quads 
Engage approach function 
Announce approach active mode 
Announce Quads 
103-LVK 

Program 103-LVK 

Engage VS and arm Altitude Hold 

Set OBS 246 to Uhhut 

Engage Heading Select and arm Approach 

Set GPS to sequencing mode when established 

Announce Uhhut 

Announce approach active mode 

Announce Oyahi 
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Appendix 2 


Script of Tasks Used for the Jet Transport Cockpit 
Automation Training Flights 


1 . Basic Data Entry and Access 

a. Access page using page button 

b. Access page using line button 

c. Find information on page 

d. Make line entry 

e. Copy and paste line entry 

f. Select page line option 

g. Scroll to next page 

2. Check Navigation Database 

a. Access Status page 

b. Check effective dates 

3. Position Initialization 

a. Enter identifier to lookup coordinates of KSFO 

b. Enter KSFO coordinates to set position 

4. Program Route 

a. Enter KSFO as origin airport 

b. Enter KLAX as destination airport 

c. Install PORTE3 departure procedure and AVE transition 

d. Install SADDE6 arrival procedure and AVE transition 

e. Install 1LS Runway 24L approach 

5. Execute modifications 

6. Performance Initialization 

a. Enter fuel on board 

b. Enter cruising altitude 

c. Enter gross weight 

7. Review Route 

a. Check Legs page 

b. Check Route page 

c. Check multifunction display 

8. Follow Route 

a. Point out the active waypoint 
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b. Report time and distance to active waypoint 

9. Direct To 

a. Program direct to assigned waypoint 

10. Add/Delete Waypoint 

a. Delete waypoint 

b. Delete route discontinuity 

c. Add waypoint 

d. Delete route discontinuity 

1 1 . Different approach or transition 

a. Select new approach 

12. Hold 

a. Select hold waypoint 

b. Program course, turns, leg length, and EFC 

c. What will happen when the airplane reaches the hold fix? 

d. Exit the hold 

13. Plan and Execute Descent 

a. Enter crossing restrictions for SYMON and BAYST 

b. How do you know when to start down? 

c. At the top-of-descent point, dial down altitude 

d. Engage vertical speed function 

e. Determine and dial the required vertical speed 

f. Determine whether or not you will meet the restriction 

g. How is the airplane maintaining this constant-rate descent? 

h. What will happen when the airplane reaches 12,000 feet? 

14. Fly Heading / Intercept Leg To 

a. Fly heading 130 

b. How do you know that HDG is engaged? 

c. Make assigned waypoint the active waypoint 

d. Program the desired intercept course 

e. Arm the Nav function to capture 

f. What will the airplane do once it reaches the course? 

15. Constant-Speed Descent 

a. Engage the Speed function 

b. Dial assigned speed 

c. How does the airplane maintain the 280 knots? 
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